The built environment is exposed to multiple aging stressors, from corrosion and fatigue to natural and anthropogenic hazards. These make our systems’ performance decline over time, raising life-cycle socioeconomic and environmental costs. Although advances in computing power and computational models have allowed us to create highly accurate digital twins to pin down the causes and effects of structural deterioration, these tools alone cannot warrant future performance: coupling them with rigorous intervention planning is paramount. This planning problem is computationally hard, due to the exponential growth of state and action combinations with the number of components and decision steps, the inherent sophistication of high-fidelity digital models, and the pervasive presence of uncertainties. Deep reinforcement learning approaches have been recently shown to be able to address these complexities at unprecedented scales. In this talk, a general reinforcement learning framework for algorithmic decision-making under uncertainty tailored to sequential intervention planning in the built environment is presented. The framework integrates uncertainty propagation, Bayesian inference, and partially observable Markov decision processes to address the forward, inverse, and control problems jointly, in a closed-loop fashion, in order to identify adaptive long-term plans. Key focus is placed on recent developments within decentralized and hierarchical multi-agent actor-critic deep reinforcement learning architectures, which provide significant scalability advantages in environments with large action spaces by relaxing and decomposing the centralized problem, respectively. Potential trade-offs of this relaxation are examined, particularly in relation to known issues in multi-agent cooperation under varying (de)centralized training and execution paradigms. The integration of fast optimization proxies for multi-component resource allocation subproblems is also discussed, with applications in inspection and maintenance settings that involve low-/high-dimensional observational data and hard/soft constraints. The significant potential of algorithmic decision-making to reduce long-term costs is demonstrated through applications in building energy retrofitting, bridge structural integrity management, and operation and maintenance of energy and transportation assets, among others.